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ABSTRACT 



The purpose of this study was to describe and interpret the 
cognition of a graduate student during information retrieval using the World 
Wide Web. The participant was a doctoral student in psychology with little 
experience using the Internet, and even less experience with the World Wide 
Web. The student performed an open search of her dissertation topic (child 
temperament) , followed by a pre-arranged task requiring her to answer several 
questions about the Civil War using only the Web to obtain information. Three 
qualitative sources of data were collected involving student cognition: 
observation, think aloud protocols, and an in-depth interview. A core 
category derived from the qualitative analysis indicated that the student's 
thoughts and perceptions centered on information overload. A systematic 
approach outlined two major dimensions of this core category: knowledge and 
personal characteristics. Within these dimensions, several subcategories were 
identified. The results are interpreted within a theory of information 
processing. Implications for continuing this line of research are provided. 
(Contains 2 tables and 13 references.) (Author/SLD) 
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Abstract 



The Intensity of Internet use in higher education, particularly the World Wide Web, has 
stimulated concern regarding how students acquire the needed skills in order to acquire, 
manage, filter, and store the vast amount of information that is accumulating daily. 

Research is beginning to focus on how students think about a search and how they use 
cognitive strategies for information retrieval. The purpose of this study was to describe and 
interpret the cognition of a graduate student during information retrieval using the World 
Wide Web. The participant was a doctoral student in psychology with little experience 
using the Internet, particularly fewer experiences with the World Wide Web. The student 
performed an open search of her dissertation topic (i.e., child temperament), followed by a 
pre-arranged task requiring the student to answer several questions regarding the Civil War 
using the only the Web to obtain the information. Three qualitative sources of data were 
collected involving student cognition: observation, think aloud protocols, and an in-depth 
interview. A core category derived from the qualitative analysis indicated that the student’s 
thoughts and perceptions centered on information overload. A systematic approach 
outlined 2 major dimensions of this core category: knowledge and personal characteristics, 
and within each dimension several subcategories were identified. The results are interpreted 
within a theory of information processing. Implications for continuing this line of research 
are provided. 
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Graduate Student Cognition During Information Retrieval Using the World Wide Web: 

A Pilot Study 

In today’s information age, student use of the Internet, particularly for educational 
purposes, has become commonplace since most schools and universities are now 
implementing the technology [1]. As a consequence, it is becoming more difficult for both 
students and teachers to keep up with the expanding knowledge base, and it will be 
especially difficult for technologically-impaired individuals to locate information, to filter 
through it, and to extract that knowledge which, for one, is necessary to remain on the 
cutting edge of professional practice. This has serious implications for training graduate 
students, who are becoming one the most frequent users of the Internet for specific 
information retrieval [2], Interestingly many graduate students and their research 
supervisors lack important “information skills.” That is, they lack skills like: (1) ability to 
identify and appraise Internet-based sources, (2) information management, (3) retrieval, 
filtering, and storing information, and (4) the ability to communicate and synthesize 
information using the Internet. One line of research that has attempted to understand this 
current problem has been from the theoretical framework of cognitive science and artificial 
intelligence [3] wherein models are generated for the purpose of designing document-based 
retrieval systems. Although such models of on-line information retrieval has applicability for 
designing computerized information systems, there exists a paucity of research, especially 
from the qualitative perspective, describing student cognition. Still, few researchers have 
investigated novice users of the Web at the doctoral level. The present investigation is a pilot 
study of one doctoral student’s cognition during information retrieval using the Internet’s 
World Wide Web. 

Review of the Literature 

The electronic library and the Internet are altering the nature of information behavior 
in academic research. It is foreshadowed that by the end of the decade, 25 million 
commercial subscribers and seven million personal connections are expected [4]. 
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Information seeking, retrieval, management, and communication of information are till 
affected by the move from traditional (i.e., the library) to information technology (IT) 
assisted methods such as the Internet [2]. One specific change is the intensification in the 
need for information skills in an increasingly complex information-rich world. For 
example, formulation of the information need must be more precise in an electronic world in 
order to construct explicit search language, and be more focused in light of increasing 
information to avoid overload [2]. This means that electronic (e.g., WEB) browsing skills 
must be learned, and to locate sources requires knowledge of access protocols for IT 
systems, that is, examining, selecting, and rejecting sources requires complex searching 
skills and an increase need to evaluate and filter out irrelevant information. 

According to Barry [2], training graduate students in information skills should take 
into account the different cognitive and learning styles of individual users and there is a need 
to develop mental models of complex information systems in the minds of users if resource 
systems like the Internet are to be effective. Hill and Hannafin [5] suggest that a variety of 
strategies are used by students when using hypermedia. They also found that student 
knowledge of the technology does have an effect on the strategies used and perceptions of 
disorientation (i.e., drowning) and level of perceived self- efficacy have an effect on the 
strategies used. 

So, it is no surprise that in designing effective interactive media like the Internet, 
cognitive psychology along with human factors, semiotics, and psycholinguistics should be 
considered [6]. For example, cognitive scientists such as Chen and Dhar [3] determined the 
cognitive processes of a sample of students during on-line document-based information 
retrieval in order to design several computational models of on-line document-based retrieval 
systems. Although this line of research in artificial intelligence is compelling, research 
examining the cognitive framework of students during Web searches is scant. 

Nevertheless, from a cognitive framework, some consensus is emerging about how 
humans think and find meaning in their world, specifically how information is used to 
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construct meaning. The active process of forming meaning from information is the task of 
the user during the information search process [7], An information search is a process of 
construction which involves the whole experience of the person, feelings, as well as 
thoughts and actions. Concentrating on cognitive aspects, Belkin, Brooks, and Oddy [8] 
describe the constructive process of information seeking in terms of the anomalous state of 
knowledge (ASK) hypothesis. To define, an information search begins with the user’s 
problem. The gap between the user’s knowledge about the problem or topic and what the 
user needs to know to solve the problem is the information need. Hence, the user’s state of 
knowledge is dynamic not static and changes during the search process. 

Information seeking and processing is not the result of a simple, linear process where 
a discrete need for information arises. First, a method of retrieval such as the World Wide 
Web is decided on to access the information, and then the information is acquired. This is 
particularly important for the graduate student because their intentions are specific, focused 
and direct. One could say that graduate students are “self-controlling - sense making” 
information seekers. Graduate students, especially those at the doctoral level, may benefit 
most from using electronic sources [2]. Databases will enable them to cover the literature 
comprehensively in their fields; the Internet can furnish them with Information on progress 
in current projects pre-publication via project home pages and preprint bulletin boards; 
electronic communication can help them to build up a network of contacts; on-line mailing 
lists will alert them to upcoming conferences and bibliographic databases will assist in 
management of large literature base. Keeping that in mind, doctoral students experience 
frustration early on when they have to spend time choosing a topic for their dissertation. 
Unable to navigate the expanding world of resources in or order to focus on a topic, doctoral 
students may search and read indiscriminately instead of selectively while evaluating 
important works and other relevant information [9]. 




6 



Cognition and the Web 5 



Statement of the Problem 

In light of the implicit nature of information skills, the incorporation of cognitive 
skills training is believed to be required to effect good information skills training. Students 
need to formulate strategies for finding information on the Internet and how to evaluate the 
information rather than focus on mechanics of Internet tools [10]. Taken a step further, when 
academic supervisors (i.e., faculty) become more skilled in electronic information handling, 
and when they are aware of the cognitive aspects of information skills, they can provide 
more active assistance in these areas to their doctoral students [2]. The impact of the Internet 
on information activity in academic research is a relatively new area of study; and existing 
research has used mainly quantitative approaches, however the domain under study is 
sufficiently problematic as to require the use of more flexible qualitative methods [1 1]. By 
utilizing a qualitative design, the purpose of the present study is to investigate graduate 
student cognition associated with Internet Web-based information retrieval. 

Method 

The Participant 

A 28 year old school psychology doctoral student named “Ann ’’attending a large 
comprehensive state university served as the participant for the study. Ann was selected 
from the graduate student body within the department of educational psychology with the 
help of a mutual colleague who new her as someone with little Internet experience. At the 
time of the investigation, Ann was in the process of beginning her doctoral dissertation, and 
had only experience with some Internet databases (e.g., e-mail, ERIC, PSYCHLIT). She 
reported having little experience with using the World Wide Web and had no information 
skills training. Ann also had no prior knowledge of the answers to any of the Civil War 
questions given to her as an information retrieval task. Since the present study focused on a 
novice user of the Internet and the Web, this made Ann an ideal candidate for study. 
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Design and Procedure 

The student and I met in a private computer lab inside the university library. The 
student began by conducting an open search of her dissertation topic (child temperament) 
using the Internet’s World Wide Web. She was instructed to verbally report her thoughts and 
perspectives as she browsed the Web. I observed her behavior and tape recorded her reports. 
Following the open search, Ann was given a sheet of paper with 5 general questions about 
certain Civil War Battles (See Table 1). These questions were written by the investigator by 
using the Web to locate basic facts about the Civil War A peer tested the stability of the 
questionnaire by performing the search himself; he was successful in answering the 
questions using the web. As in Ann’s open search of child temperament, I observed her 
behavior and verbal reports were recorded. I chose a design wherein the participant would 
conduct both an open search of a topic of her choice and a structured task to answer pre- 
arranged questions for two reasons. For one, since Ann was a novice Web user, I wanted 
her to get comfortable with searching the Web at the same time collect data regarding her 
cognition during the search of her topic of interest. Second, I wanted to obtain her cognition 
during a structured task. I was interested in similarities between the two searches in order to 
obtain a complete picture of her cognitive framework. 

[Insert table one here] 

Data Collection 

Three sources of qualitative data were collected from the participant. The first were 
verbal reports of Ann’s thoughts while searching the Web. They were obtained using a think 
aloud protocol strategy. Think aloud techniques have been used extensively and are believed 
to reflect more immediate cognitive processes due to reduced processing time and less 
distortion in the structure of processing [12]. These protocols were recorded onto an audio 
cassette and later transcribed. I assumed the role of non-participant observer wherein I 
remained present to observe and interact with the participant when necessary (e.g., give 
instructions or clarify). The second source of qualitative data was a semi structured 
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interview. Ann was asked several questions regarding her history of computer and Internet 
use, her methods for information seeking, and questions about her thoughts during her Web 
searches. Some questions were geared toward reflective information, such as “what 
strategies did you use during your Web search?” and “At what point did you get frustrated 
when you couldn’t find a specific piece of information, and describe that thought?” The 
interview was recorded and transcribed. Approval for this investigation was granted on 
behalf of the institutional Review Board. The participant was given informed consent about 
her right to withdrawal at any time, and she was assured that her anonymity would be 
upheld. 

Data Analysis 

Both open and axial coding [13] were used to develop categories. The student’s data 
sources underwent open coding such that events, ideas, and actions were labeled as concepts 
and sorted into broad, tentative categories. Categories were compared for similarities and 
differences to increase their clarity and distinctiveness. A single core category pertaining to 
student cognition was derived from the data. Because information retrieval was the 
expressed goal of the student, the student’s responses that centered on her most significant 
problem in retrieval provided insight into this core category. In addition to the consistent 
focus in these responses, analysis of the think aloud protocol confirmed that a basic issue 
was of central concern to the student. Last, to ensure credibility of the findings, one peer 
reviewer briefly analyzed the data and concurred with my findings and conclusions. 

Results 

The primary research question dealt with describing and interpreting graduate student 
perspectives and thoughts during the process of information retrieval using the World Wide 
Web. The following section addresses this question using conventions adopted from Strauss 
and Corbin [13]. Table two summarizes the relationship between the core category and its 
dimensions derived in the study. 

[Insert T able two here] 
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Core category 

Ann’s cognition during her search for information on the WEB were characterized by 
one central theme involving information overload. The links between this category and its 
dimensions and subcategories are presented below. 

Dimensions underlying information overload 

Two specific dimensions of the core category are (a) knowledge regarding computer 
and Internet use, specifically mechanical and information skills; and (2) personal 
characteristics/ experience the student brought to the WEB-based search process. 

The first dimension of the core category was knowledge and was defined as the 
ability to understand and apply skills using the computer, specifically the Internet’s 
capabilities, to retrieve specific information. There were two subcategories of knowledge 
derived from the data. The first was mechanical skills. This involves Ann’s ability to 
maneuver or “browse” the Web using various tools within a navigator, such as scroll icons 
and arrow keys to toggle from page to page. For example, Ann verbally reported from one 
think aloud session: 

“How do 1 reach a site.. oh okay. I just click on the words that are underline in blue 
and 1 am sent automatically to that site... [Ann reaches the site].. I can’t seem to find 
it.. .oh. .It must be at the bottom under this passage, but how can I get there?.. I can 
scroll down using the arrow on the side of the window. I need to understand this 
navigator system better! ” 

Ann also stated in the interview that , 

“I had always had a fear of computers... and the technology changes so much it is 
hard for me to keep up with just learning how to maneuver the web using the tools.” 

It is clear that information overload for Ann at times may be associated with her lack of 
knowledge in areas that involve basic tool use. She reports that “If only I had more 
sophisticated computer skills it wouldn’t be so hard to find things.” 
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Another aspect of the knowledge dimension was the level of sophistication involving 
Ann’s information skills. Information skills are defined as the ability to retrieve, filter and 
store relevant information, especially an ability to determine good and relevant from poor or 
irrelevant information. Here, Ann reported during her search to answer one specific Civil 
War question, 

“Seeing a bunch of those things [Civil War Web sites] listed and seeing them tied 
together. I thought I can just pick a place to start and IT will eventually run me right 
where I need to go. ” 

Ann states many times in the interview that browsing can eventually move you where you 

want to go. This has direct implication for information overload because as she browsed 

using general topics like “Civil War” she often found herself looping around the same sites 

and so she simply focused her search on these sites, without even choosing different terms 

to limit her search. Another impact information skills has on information overload was 

exemplified when Ann tried to answer one Civil War question about whether or not George 

McClellan won a battle he was to have found. She expected a web site would produce the 

answer explicitly, and so she focused her search on a web site that would “spell out” the 

answer. To her dismay, she was not able to find such a site, but later while answering a 

different question happened to stumble upon a passage that indirectly answered the question 

(the passage stated that McClellan was fired in a certain year which described a battle he 

fought. One verbal report reflects her lack of information skills: 

“This Web site told me a year and battle he fought but not the outcome... here it is 
stated funny.. .it says that another general took his place once he was fired. So, I 
assume that he lost the battle! but I feel I can’t make the assumption.. .it seems I need 
to check out other sites that can give me a clearer answer.. ..I want to find a passage 
that says he ways DEFEATED and then FIRED. ” 
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It is apparent that Ann’s lack of Information skills training may be associated with 
information overload, especially during her search for certain answers to one or two of the 
Civil War questions. 

The second dimension of Information overload derived from the data was Ann’s 
personal characteristics and experience which she brought to the search. This dimension is 
defined by 3 subcategories: (1) personal needs, with information importance placed on a 
continuum from general to specific importance, (2) tolerance for information ambiguity and 
redundancy, and (3) patience and time management. 

Ann’s personal need regarding information was mostly reflected in her open search 
of her dissertation topic of child temperament. She explained that her need was very specific 
in that she was anticipating professional research articles and professional organizations on- 
line. What she found was a divergent amount of information on the subject, ranging from 
one review of a current study related to her dissertation topic to a commercial on-line 
questionnaire designed to assess “your child’s temperament. ” Ann stated in one verbal 
report: 

“Since I am doing my dissertation on child temperament I would like to see journal 
articles and references and such like you get in the library... Wow.. this site looks like 
a very colorful commercial questionnaire you can fill out right on-line to assess my 
child’s temperament level... odd.. [continues searching] I also notice some sites are 
about commercial books.. I am thinking this Web provides very commercial 
information.. designed for the layperson.” 

Given this line of evidence, it appears Ann is driven by specific information needs, 
seemingly to the point she was biased in her appraisal of the Web (she thought it would give 
her “scientific and professional” information foremost. She describes the Web as being 
commercial and “for the layperson” and when her specific information needs are hidden 
within a web of vast general information, Ann experienced a sense of “drowning”. She 
stated in the interview. 
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“only tid bits of information are available on my topic, so I wouldn’t start using this 
Web more now because it just wasn’t reinforcing enough.” 

Another line of evidence that supported how her need for specific information led her to 

experience information overload stemmed from her search for specific Civil War facts. Her 

need to locate very specific facts at times frustrated her because, through her method of 

searching, only obtained general sites which did not provide the facts explicitly. 

The second subcategory of Ann’s personal characteristic dimension was her degree 

of tolerance for information ambiguity and redundancy. Given the previous finding that Ann 

had very specific information needs, her tolerance for “commercial” and ambiguous 

information led her to experience overload. When asked about this in the interview, she 

stated that most can be attributed to the way the Web is structured. When asked how she 

would structure the Web if she had control, she stated: 

“I would have something like a “table of contents” popping up immediately showing 
the different sections of the general topic. I would have it divided by area 
(professional reports, organizations, etc.).” 

She believes this would be good for all searches of a general topic (like temperament) 
because one would not have to deal with redundant information and confusing Web sites. 
When she was asked why a table of contents for general searches would be helpful she 
replied, 

“It [table of contents] narrows it down much faster. I guess most people can browse 
and scroll. But for someone like me who wants specific information I would like to 
avoid the junk and be able to access it quicker.” 

Even observing Ann’s behavior during her searches reflected her frustration with ambiguous 
information. She would twitch and one time yelled out “I know what I need is there, I 
know I am not finding it.. .it is somewhere in this crud!”. A final piece of evidence derived 
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from the interview came at the end when she was asked, “At what point did you feel you 

were drowning during the search process?” Her reply was, 

“Well. .there were some questions regarding the Civil War that were hard to 
answer.. it was like you know what you want but you just can’t find it. I guess part 
of it was like believing that the Web can read your mind about the information you 
want. ” 

Here, it clear the broad nature of the Web and all its ambiguity has gotten the best of her (it 
knows what she wants??). In a sense since she has not acquired sophisticated search 
strategies and information skills, she has not taken full control of her search; she was at 
times a bit lost in the web search. 

The third subcategory of.Ann’ personal dimension was her level of patience with the 

search process and time management. She replied at one point in the interview, 

“I remember being in somewhat of a hurry toward the end of my search and having 
to scroll through redundant and confusing information took its toll on me time wise.” 

Ann places a huge emphasis on time and patience during a search. She compares it to going 
to the school library at 1 1:30 when you know it closes at midnight. She views the Web as a 
“different beast” in that if you are on the Web and you have time and relaxed the search can 
be more fruitful. She also stated that, 

“A lot of the drowning is probably due to my personality... I panic in time crunches.” 
Since she was hurried to finish, she placed less emphasis on changing her search strategy 
and more on scrolling around the same two Web sites to get what she needed (for example, 
trying to determine if George McClellan won the battle in 1861). 

Conclusion 

The primary purpose of this pilot study was to investigate the focus of a graduate 
student’s cognition within a Web-based information retrieval setting. The contextual factors 
influencing student cognitive activity was examined. One doctoral level graduate student in 
psychology with few Internet experiences served as the participant. 
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The major finding of this study involves the fundamental concern and experience of 
information overload while retrieving information from the Web. The emergence of 
information overload as a core category was supported by the extensive link of this category 
with various subcategories. Furthermore, these subcategories where segregated into two 
dimensions of the core category: knowledge and personal characteristics. 

Under the knowledge dimension, the degree to which the novice user can manipulate 
the many capabilities of the Web browser (e.g., knowledge of the different search engines 
like Y ahoo) to conduct searches has an impact on the extent to which the searcher “drowns” 
during the search process. A novice user like Ann for instance did not use different search 
engines and had difficulty toggling from screen to screen, thus limited her strategies and led 
her to experience more overload. In addition, the extent to which Information skills can be 
applied will determine level of information overload. The results of Ann’s profile support 
the findings of Barry [2] that doctoral students lack the imperative information skills 
necessary to avoid drowning. Also, the knowledge dimension derived from the data 
supports the findings of Hill and Hannafin [5] in that student knowledge does have an effect 
on the strategies used. 

The second dimension of the core category, personal characteristics, also affected 
information overload. Graduate students like Ann who are beginning a dissertation have 
very specific needs and interests, coupled with the fact that time management is important. 
These students may become impatient with an elongated search and so the need to take 
control of the Web search (and not just browse) is recommended. Many graduate students at 
this level, like Ann, become frustrated with information that isn’t relevant to the topic of 
interest. The net result is information overload. Also, graduate students may, given their 
lack of information skill training, be susceptible to information drowning when they are 
faced with a lot of redundant and ambiguous information (e.g., commercial Web sites). 
Ann’s need for professional Web sites and on-line articles led to assume the Web was 
organized around that need. 
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The best avenue of interpretation of the present findings appears to be through a 
theory of information processing. Information processing theory states that the mind is 
analogous to a computer, and it thinks in terms of a cycle of input of information (based on 
perception), storing of information, and output (based on retrieval of stored information). 
Disruption in this cycle occur when too much information is assimilated at one time - mainly 
because the individual cannot successfully filter and manage the information. One way to 
explain the information overload focus of Ann’s cognition is to examine the way information 
is retrieved and processed during a search. A method, such as specific Internet tools like the 
World Wide Web, was decided on to access the information, and then the information was 
acquired. Ann very seldom used any search strategies other than using generic terms to 
locate Web sites (for the Civil War exercise). Search strategies are important; one should 
learn to choose the “right” terms before logging onto the Web. Merely browsing sites 
haphazardly is not recommended. 

A skilled user should use cognitive strategies that deal with language manipulation, 
that is, knowing how to narrow down to specific information and filter through the rest 
using the best search terms. For example, instead of Ann using “Civil War” as a search term 
to find three battles fought in 1863, she could have used terms like “timeline”, “chronology 
and battles” as more strategic semiotics. Hence, in light of information processing theory, 
the key is to leam how to filter and manage the pertinent information taken from the Web if 
disruption in the processing cycle due to overload is to be avoided. 

Ultimately, the goal of information process is not only to acquire information, but to 
form meaning from it. The active process of forming meaning from information is the task of 
the user during the information search process [7], An information search is a process of 
construction which involves the whole experience of the person, feelings, as well as 
thoughts and actions. Ann’s personal characteristics, especially her lack of tolerance toward 
“commercial junk” and her impatience support this theory. All in all, formulation of the 
information need must be more precise in an electronic world in order to construct explicit 



O 

ERLC 



16 



Cognition and the Web 15 



search language, and be more focused in light of increasing information to avoid overload 
[2], This means that electronic (e.g., WEB) browsing skills must be learned, and to locate 
sources requires knowledge of access protocols for IT systems, that is, examining, 
selecting, and rejecting sources requires complex searching skills and an increase need to 
evaluate and filter out irrelevant information. 

Implications for further research 

The findings of this pilot study do lend support for prior research [2,7] and provides 
some interesting insights into information overload as a core dimension of cognition held by 
the student during Web-based searching. However, the results of this Study are not 
conclusive and should be corroborated utilizing a more representitve sample. 

The advent of the electronic library heralds possibilities for significant change in the 
future world of academic research. The traditional academic locus of information activity, 
the library, has now been supplemented by the option of accessing information from a desk 
top computer, and futuristic convention maintains that the traditional methods like that of the 
library in its current form will eventually become redundant [1 1], It is apparent that as the 
world becomes more digitized, information will take on newer forms and continue to 
expand, making it difficult for all academicians to stay abreast. The major implication rests 
on a need for more Information skills training, specifically during the high school or 
undergraduate years. If librarians and faculty can assure training early enough, by the time 
students enter graduate school, use of the Internet and its capabilities will not be 
burdensome. The present findings suggest that cognitive strategies for effective searches 
wherein information overload can be minimized should take into account the user’s level of 
knowledge about computers and information skills, in conjunction with the individual 
personality of the user. 
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Table 1 

A task to retrieve information regarding some basic facts of the American Civil War. 

1. In what year was the first battle of Bull Run? 

2. Name 2 battles that occurred in 1863. 

3. Who was George B. McClellan? 

Name one battle he fought. 

Did he win that battle? 

4. In what year was the expedition and capture of New Orleans? 

What forts were involved? 

Name one principal commander (either Union or Confederate). 

5. Briefly describe the events of the battle of Fort Sumpter. 
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Table 2 

Relationship between the core category and its dimensions 



Core category Dimensions 


Subcategories 


Information Knowledge 

overload 


Mechanical skills 
Information skills 


Personal 

characteristics 


Personal needs (general - specific) 
Tolerance for ambiguity and redundancy 
Patience and time management 
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